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METHOD AND SYSTEM FOR OVERLOADED ARRAY PROCESSING 

RELATED APPLICATION 

This application claims the benefit of U.S. 
provisional application Serial No. 60/204,117, filed May 
15, 2 000, entitled SPATIALLY REDUCED SEARCH JOINT 
5 DETECTION FOR OVERLOADED ARRAY PROCESSING. 

TECHNICAL FIELD OF THE INVENTION 

This invention relates generally to the field of 
wireless communications and more specifically to a method 
10 and system for overloaded array processing. 
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BACKGROUND OF THE INVENTION 

Many antenna arrays such as airborne antenna arrays 
have a limited number of antenna elements due to size and 
weight restrictions, but receive multiple signals due to 
5 their high altitude and resulting large field of view. 
An antenna array becomes overloaded when the number of 
signals received by the array exceeds the number of 
antenna elements of the array. Techniques for overloaded 
array processing, however, require a large amount of 
10 computational power and time. Consequently, overloaded 
array processing has posed challenges in the field of 
wireless communications. 
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SUMMARY OF THE INVENTION 

While known approaches have provided improvements 
over prior approaches, the challenges in the field of 
wireless communications have continued to increase with 
5 demands for more and better techniques having greater 
effectiveness. Therefore, a need has arisen for a new 
method and system for overloaded array processing. 

In accordance with the present invention, a method 
and system for overloaded array processing are provided 
10 that substantially eliminate or reduce the disadvantages 
and problems associated with previously developed systems 
and methods . 

According to one embodiment of the present 
invention, a method for array processing is disclosed. 
15 Signals transmitting a symbol set are received. A 
dominant signal set for each signal is determined. The 
dominant signal set for a signal includes dominant 
signals that interfere with the signal. A trellis is 
constructed from the dominant signal sets. The trellis 

2 0 includes paths that represent possible symbol sets. An 

optimal path from the trellis is selected, and the symbol 
set represented by the optimal path is determined. 

Certain embodiments of the invention may provide 
numerous technical advantages. A technical advantage of 
25 one embodiment is that a spatial-trellis is used to 
describe possible symbol sets transmitted by spatially 
diverse signals. The trellis may significantly reduce 
the required search with little performance degradation 
over the optimal solution. Another technical advantage 

3 0 of one embodiment is that a sparsity pattern is used to 

identify signals that produce dominant interference and 
to eliminate the rest of the signals. The sparsity 
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pattern may provide an efficient method for constructing 
a trellis of reduced complexity, thus decreasing 
computational time and power. Another technical 

advantage of one embodiment is that an efficient search 
5 process is used to locate an approximate least cost path 
through the trellis. The least cost path identifies an 
optimal symbol set. 

Other technical advantages are readily apparent to 
one skilled in the art from the following figures, 
10 descriptions, and claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention and for further features and advantages, 
reference is now made to the following description, taken 
5 in conjunction with the accompanying drawings, in which: 

FIGURE 1 is a block diagram illustrating a system 
for array processing; 

FIGURE 2 illustrates beams formed from eight signals 
impinging on five antenna elements; 
10 FIGURE 3 is an interference diagram illustrating 

interference that each signal of FIGURE 2 experiences 
from the other signals; 

FIGURE 4 is a sparsity pattern for the interference 
diagram of FIGURE 3 ; 
15 FIGURE 5 is a trellis generated from the sparsity 

pattern of FIGURE 4; 

FIGURE 6 is a flowchart illustrating a method for 
array processing; 

FIGURE 7 is a flowchart illustrating a method for 
2 0 constructing the trellis of FIGURE 5 from the sparsity 
pattern of FIGURE 4; and 

FIGURE 8 is a flow chart illustrating one method for 
determining an optimal path of the trellis of FIGURE 5. 
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DETAILED DESCRIPTION OF THE DRAWINGS 

Embodiments of the present invention and its 
advantages are best understood by referring to FIGURES 1 
through 8 of the drawings, like numerals being used for 
5 like and corresponding parts of the various drawings. 

FIGURE 1 is a block diagram illustrating a system 
100 for array processing. System 10 0 includes an antenna 
array 113 of radius R that has antenna elements 110 that 
receive signals p lr p 2 , ■ ■ ■ , Pd- Signals p d transmit 

10 symbols to system 100, which processes the signals p d to 
extract the symbols. Signals p d may be, for example, co- 
channel, complex pulse -amplitude modulated (PAM) signals 
that are symbol - synchronous , or have perfectly aligned 
symbols. The number D of signals p d received by antenna 

15 elements 110 may be greater than the number M of antenna 
elements 110 such that antenna array 113 is overloaded. 
Antenna elements 110 generate antenna data x lf x 2 , . . . , 
x m , . . . , x M in response to detecting signals p d . 

The relationship between the antenna data and 

20 symbols for system 100 is described by Equation (1) : 

x[n] = As[n] + z[n] (1) 
where vector s[n] = [sjn] s 2 [n] ■■■ s D [n]\ is a Dxl vector of 
the nth symbol set transmitted by the signals, and s d [n] 
is the symbol set transmitted by signal p d . Each s d [n] is 

2 5 independent and drawn equally likely from an alphabet of 
symbols. Vector x [n] = [x 2 [n] x 2 [n] . . . x M [n]J is an Mx.1 
vector of antenna data generated in response to the nth 
received signals. Matrix A is the composite array 
response for the signals, where the dth column of matrix 

30 A is the steering vector for the dth signal. If signals 
p d are at zero elevation. If the array has a circular 
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geometry then the steering vector for antenna array 113 
is given by Equation (2) : 

a{e d ) = [a x -a M Y,a m = expj^- jK^sin^ - ^^T^ )) ^ 

where the constant K x =2xR/A, angle 0 d is the azimuthal 
angle of arrival, X is the wavelength of the carrier 
frequency, and T represents vector transpose. Vector 
z[n] is an Mxl vector representing temporally 
uncorrelated noise with zero mean and autocorrelation 
O zz = E[z[n]z[n] H ] , where H represents the Hermetian 
transpose, or the conj ugate- transpose , of a matrix. 

System 100 utilizes a modified form of a joint 
maximum likelihood (JML) criterion to identify the most 
likely symbol set transmitted by signals p d . The modified 
form represents a cost function, which is optimized to 
determine the most likely symbol set. If no inter-symbol 
interference is present, a joint maximum likelihood (JML) 
criterion for detecting signals p d impinging on antenna 
array 113 is described by Equation (3) : 

s[n] = arg min(x[n] - As) h O^(jc[j2] - As) , . 

The JML criterion is modified to produce a cost function. 
Equation (3) is simplified by expanding products and 
dropping terms independent of the optimization to yield 
Equation (4) : 

s[n] = arg min\s H A H O^As - 2 Re fp" 1 As)" *_}} ( 4 ) 

where Re represents the real part of a number. A DxD 
matrix H and DkI vector y are defined by Equations (5) : 

H /f H = A H 0^ 1 A, (5) 
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Matrix H may be written as a spectral square root 
described by Equation (6) : 

H = (a"$>) (1/2) (6) 
Matrix H may be used as an interference matrix that 
5 describes the energy that one signal receives from other 
signals. Matrix H may be selected such that the 
interference represented by each row of matrix H is 
concentrated . 

Equation (4) is rewritten as a cost function as 
10 defined by Equation (7) : 

s[n] = arg s min||y - HsJ 

A, 2 (7) 

- argjnin£|y[d]-y[d][ 

d=l 

where h[d] is the dth row of H, and y[d] =h[d] s. 

Vector y is selected to complete the square of the 
cost function as described by Equation (7) such that when 
15 the quantities on the right side of Equation (7) and the 
matrix H are known, for example, vector y may be defined 
by Equation (8) : 

y = ((H H ) + A H "£)k = WJC (8) 

2 0 where + represents the penrose psuedo- inverse of a 
matrix . 

A module set 112 uses Equations (5) through (8) to 
process antenna data x m to identify the symbol sets s[n] 
that are communicated by the signals p d . The modules of 

2 5 module set 112 may include hardware, software, or any 
suitable configuration of hardware and software. Module 
set 112 includes a beam former module 12 0, an 
interference module 122, a sparsity pattern module 124, a 
trellis module 126, a path module 128, and a symbol 

30 module 130. Beam former module 120 applies a beam former 
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matrix W, as defined by Equation (8) , to antenna data x m 
to yield beam outputs y d . Matrix W is a multi- input, 
multi -output (MI MO) beam- former matrix since each row of 
matrix W performs an implicit beamforming operation and 
5 all rows together preserve the maximum- likelihood 
criterion. For certain antenna arrays, matrix IV focuses 
the dth beam on the dth user such that every user has 
dominant energy in at least one beam, thus reducing the 
complexity of overloaded array processing. A 

10 representation of the beams is illustrated in FIGURE 2. 
FIGURE 2 illustrates beams b x , b 2 , . . . , b 8 formed from 
eight signals p d impinging on five antenna elements 110. 
The angles of arrival for signals p lr p 2 , • ■ ■ , Ps are 
12°, 64°, 115°, 171°, 186°, 261°, 278°, and 353°, 

15 respectively. 

Interference module 122 uses interference matrix H, 
as defined by Equation (5) , and beam outputs y k to 
determine interference among signals p d . FIGURE 3 is an 
interference diagram 15 0 constructed from matrix H, which 

2 0 illustrates interference that each signal p d of FIGURE 2 
experiences from the other signals. The rows of 

interference diagram 15 0 represent the energy that an 
affected signal receives from interfering signals, and 
the columns represent the interfering signals. For 

2 5 example, the first row represents the energy that signal 
pi receives from signals p lf p 2 , • • •/ Ps- Interference 
diagram 150 indicates that signal p ± receives the most 
energy from itself and from neighboring signals p 2 and p s . 
Sparsity pattern module 124 generates a sparsity 

30 pattern from interference diagram 150. The sparsity 
pattern is a matrix of ones and zeros, where the ones 
represent dominant signals, or signals that produce 
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interference with high levels of energy, and the zeros 
represent nondominant signals. The rows of the sparsity 
pattern represent the signals affected by the interfering 
signals, and the columns represent the interfering 
signals. The dominant signals from an affected signal 
form a dominant signal set U for that signal . 

An example of a sparsity pattern 152 for 
interference diagram 150 is illustrated in FIGURE 4. The 
rows of sparsity pattern 152 correspond to the rows of 
interference diagram 150, and the columns of sparsity 
pattern 152 correspond to the columns of interference 
diagram 150. The dominant signal set U for signal pi, 
according to sparsity pattern 152, is U = {si, s 2 , s 3 , s 8 J . 
Dark squares represent ones, and white squares represent 
zeros. For each signal, the sparsity pattern selects the 
dominant signals that interfere most with the signal, and 
eliminates the nondominant signals. Thus, the sparsity 
pattern reduces complexity, allowing for more efficient 
array processing. 

If an optimal sparsity pattern cannot be readily 
determined, a sub-optimal sparsity pattern may be 
calculated according to a ratio RP of a desired signal's 
power in the beam former output y [d] to the power of the 
signals outside the dominant signal set. The ratio RP 
may be defined by Equation (9) : 

_ g[|h[d, d]s[d]l 2 ] _ \h[d, d] i 2 



2 h[d, u]s[u] 

\ueu[d] I 



X>[d,u]| 



where E represents the expected value. The sparsity 
30 pattern is generated by selecting the smallest number of 



ATTORNEY'S DOCKET 
064749 . 0134 



PATENT APPLICATION 



11 

contiguous signals that satisfy a predetermined ratio RP 
for each affected signal. 

Trellis module 126 generates a trellis from the 
sparsity pattern. A trellis represents possible symbol 
5 sets. FIGURE 5 illustrates a spatial trellis 160 
generated from sparsity pattern 152. A method for 

generating trellis 160 from sparsity pattern 152 is 
described in more detail in connection with FIGURE 7. 
Path module 128 searches trellis 160 to determine an 

10 optimal path of trellis 160 that optimizes the cost 
function as defined by Equation (7) . The optimal path of 
trellis 160 represents a most likely symbol set s[n] 
transmitted by signals p d . Symbol module 13 0 extracts the 
symbol set s [n] from the optimal path. A method for 

15 determining the optimal path and extracting the symbol 
set is described in more detail in connection with 
FIGURES 6 through 8. 

FIGURE 6 is a flowchart illustrating a method for 
array processing. The method generates a trellis that 

20 represents the possible symbol sets s [n] transmitted by 
signals p d . A search is performed on the trellis in order 
to determine the symbol set s [n] that satisfies the 
maximum likelihood criterion as defined by Equation (3) . 

The method begins at step 210, where antenna 

25 elements 110 receive signals p d and output antenna data 
x m . At step 212, beam former 120 determines beam output 
y d from antenna data x m using beam former matrix W as 
defined by Equation (8) . FIGURE 2 illustrates beams 
described by beam outputs y d . At step 214, the 

3 0 interference between beams is determined. Interference 
module 122 determines the interference by applying matrix 
H as defined by Equation (5) to beam output y d . 
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Interference diagram 150 of FIGURE 3 illustrates the 
interference among beams. At step 216, sparsity pattern 
152 of FIGURE 4 is constructed from interference matrix 
H . Sparsity pattern 152 represents the dominant signals. 

A trellis is constructed using from sparsity pattern 
152 at step 220. A method for constructing a trellis is 
described in connection with FIGURE 7. 

FIGURE 7 is a flowchart illustrating a method for 
constructing trellis 160 from sparsity pattern 152. The 
method begins at step 230 to determine stages 162 of 
trellis 160. A stage 162 represents a signal affected by 
interfering signals, and corresponds to a row of sparsity 
pattern 152. Stages 1, 2, . . .,8 represent signals p lf 
p 2r . . . , p 8 , respectively, affected by interference from 
interfering signals. 

At step 232, state sets 164 for stages 162 are 
determined. A state set 164 for a stage 162 (see FIGURE 
5) represents selected dominant signals interfering with 
the signal represented by the stage 162. State sets 164 
are selected from the dominant signal sets U of sparsity 
pattern 152. If the dominant signal sets U of sparsity 
pattern 152 satisfy U e [d] c U e [d -l]u U e [d +1] with modulo-D 
indexing, the resulting trellis 160 is called a "tail- 
biting trellis." State sets a[d] for the dth state may 
be determined from the dominant signal sets U using 
Equation (10) : 



That is, the dth stage of trellis 160, is described by 
Equation (11) : 

tr[d] u a[d + 1] = (U[d - l] n U[d] ) kj (U[d] n U[d + 1] ) 

= u[d] 11 




(10) 
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For example, state set (s S/ s 2 ) is determined for stage 1. 
At step 234, value sets 166 for a stage 162 are 
determined. Value sets 166 of stage 162 include all 
possible values that the signals, or states, of a state 
set of the stage 162 may have. For example, values 
(1,1), (-1,1), (1,-1), and (-1,-1) are determined for 
state (s 8 , Si) . 

At step 23 6, the method determines whether there is 
a next stage. If there is a next stage, the method 
returns to step 2 34 to determine the value sets for the 
stage. If there is no next stage, the method proceeds to 
step 23 8 to determine the valid transitions for a stage. 
Valid transitions are transitions between stages for 
which the corresponding states have the same value. For 
example, a transition from state set (s S/ s 2 ) with values 
(1,1) to state set (s 2 , s 2 , s 3 ) with values (1,1,1) is 
valid, because state s z has the same value 1 at both 
stages. The transition from state set (s 8/ s 2 ) with 
values (1,1) to state set (s lf s 2 , s 3 ) with values 
(-1,1,1) is not valid because s 2 has value 1 at stage 1 
and value -1 at stage 2. After determining the valid 
transitions, the method terminates. 

Returning to FIGURE 6, at step 222, an optimal path 
of trellis 160 is calculated. The optimal path is the 
least cost path for the cost function defined by Equation 
(7) . A method for calculating the optimal path is 
described in connection with FIGURE 8. 

FIGURE 8 is a flow chart illustrating one method for 
determining an optimal path of trellis 160. The method 
selects an optimal path by calculating a cumulative cost 
for each path through the stages of trellis 160. 
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The method begins at step 250, where trellis 160 is 
initialized. Trellis 160 is initialized by selecting a 
starting stage as a current stage, for example, stage 1, 
and setting the cumulative costs for all value sets at 
5 zero. At step 452, the costs of transitions from value 
sets of the current stage to one value set of a next 
stage are calculated using Equation (7) . For example, 
the costs of transitions from (1,1) at stage 1 to (1,1,1) 
at stage 2 and from (-1,1) at stage 1 to (-1,1,1) at 

10 stage 2 are calculated. The least cost transition is 
selected as a survivor at step 254. For example, the 
transition from (-1,1) to (1,1,1) is selected as the 
survivor. At step 2 56, the transition cost of the 
survivor is added to the cumulative cost of the value set 

15 at the next stage. For example, the transition cost is 
added to the cumulative cost of (1,1,1) at stage 2. 

At step 258, the method determines whether there is 
another value set of the next stage. If there is another 
value set, the method returns to step 252 in order to 

20 calculate the cost of the transitions from the value sets 
of the current stage to the value set of the next stage. 
For example, the costs of the transitions from (1,1) to 
(-1,1,1) and from (-1,-1) to (-1,1,1) are calculated. 

If there is not another value set at step 258, the 

25 method proceeds to step 260 to determine whether there is 
a next stage. Since trellis 160 is a tail-biting 
trellis, the method may proceed around the trellis 
through any number of stages. For example, the method 
may process stages 1 through 8, and then process stages 1 

3 0 through 8 again. As more stages are processed, the 
accuracy of the search increases. The number of stages 
to be processed may be determined according to the 
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desired accuracy and available computational power and 
time. If there is a next stage, the method returns to 
step 252 to calculate the transition costs at the next 
stage. If there is no next stage at step 260, the method 
5 proceeds to step 2 62 to determine the least cumulative 
cost. For example, a path ending at stage 8 with an 
endpoint value set (1, -1) has the least cumulative cost. 

The least cost path is determined at step 2 64. The 
least cost path is determined by tracing back from the 

10 endpoint value set with the least cumulative cost through 
the survivors. For example, a path with value sets for 
stages 1 through 8 is represented by [(1,1) (1,-1,1) 
(1,1) (1,-1) (-1,1) (1,1,-1) (1,-1) (-1,1) (1,-1)]. 
After determining the least cost path, the method 

15 terminates. 

Returning to FIGURE 6, the symbols are determined 
from the least cost path at step 224. The least cost 
path lists the symbols of the symbol set, for example, 
[(1,1) (1,-1,1) (1,1) (1,-1) (-1,1) (1,1,-1) (1,-1) 

20 (-1,1) (1,-1)]. Thus, the method identifies a symbol set 
that satisfies the maximum likelihood criterion as 
defined by Equation (3) . After determining the symbols, 
the method terminates . 

Certain embodiments of the invention may provide 

25 numerous technical advantages. A technical advantage of 
one embodiment is that a trellis is used to describe 
possible symbol sets transmitted by spatially diverse 
signals. The trellis is searched to efficiently identify 
an optimal symbol set. Another technical advantage of 

3 0 one embodiment is that a sparsity pattern is used to 
identify signals that produce dominant interference and 
to eliminate the rest of the signals. The sparsity 
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pattern reduces the complexity of the array processing, 
thus decreasing computational time and power. Another 
technical advantage of one embodiment is that an 
efficient search process is used to locate a least cost 
5 path of the trellis. The least cost path identifies an 
optimal symbol set. 

Although an embodiment of the invention and its 
advantages are described in detail, a person skilled in 
the art could make various alterations, additions, and 
10 omissions without departing from the spirit and scope of 
the present invention as defined by the appended claims. 



